{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "whjsJasuhstV"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_01_5_prompt_engineering.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "euOZxlIMhstX"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 1: Course Overview**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d4Yov72PhstY"
   },
   "source": [
    "# Module 1 Material\n",
    "\n",
    "* Part 1.1: Course Overview [[Video]](https://www.youtube.com/watch?v=OVS-6s20Ms0) [[Notebook]](t81_559_class_01_1_overview.ipynb)\n",
    "* Part 1.2: Generative AI Overview [[Video]](https://www.youtube.com/watch?v=ohmPaSsKhMs) [[Notebook]](t81_559_class_01_2_genai.ipynb)\n",
    "* Part 1.3: Introduction to OpenAI [[Video]](https://www.youtube.com/watch?v=C2xyi2Cq-bU) [[Notebook]](t81_559_class_01_3_openai.ipynb)\n",
    "* Part 1.4: Introduction to LangChain [[Video]](https://www.youtube.com/watch?v=qQI5AhaKxuI) [[Notebook]](t81_559_class_01_4_langchain.ipynb)\n",
    "* **Part 1.5: Prompt Engineering** [[Video]](https://www.youtube.com/watch?v=_Uot1i5sIXo) [[Notebook]](t81_559_class_01_5_prompt_engineering.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AcAUP0c3hstY"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running and maps Google Drive if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xsI496h5hstZ",
    "outputId": "4c463fae-e29a-422e-970d-6662e62c1f9d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: using Google CoLab\n",
      "Requirement already satisfied: langchain in /usr/local/lib/python3.10/dist-packages (0.1.14)\n",
      "Requirement already satisfied: langchain_openai in /usr/local/lib/python3.10/dist-packages (0.1.1)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.29)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.3)\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
      "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.6.4)\n",
      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.33)\n",
      "Requirement already satisfied: langchain-community<0.1,>=0.0.30 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.0.31)\n",
      "Requirement already satisfied: langchain-core<0.2.0,>=0.1.37 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.1.40)\n",
      "Requirement already satisfied: langchain-text-splitters<0.1,>=0.0.1 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.0.1)\n",
      "Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.1.40)\n",
      "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.6.4)\n",
      "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
      "Requirement already satisfied: openai<2.0.0,>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from langchain_openai) (1.16.2)\n",
      "Requirement already satisfied: tiktoken<1,>=0.5.2 in /usr/local/lib/python3.10/dist-packages (from langchain_openai) (0.6.0)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (3.21.1)\n",
      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (0.9.0)\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.10/dist-packages (from jsonpatch<2.0,>=1.33->langchain) (2.4)\n",
      "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2.0,>=0.1.37->langchain) (23.2)\n",
      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /usr/local/lib/python3.10/dist-packages (from langsmith<0.2.0,>=0.1.17->langchain) (3.10.0)\n",
      "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (3.7.1)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.7.0)\n",
      "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (0.27.0)\n",
      "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.3.1)\n",
      "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.66.2)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.10.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.6.0)\n",
      "Requirement already satisfied: pydantic-core==2.16.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.16.3)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.6)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
      "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<1,>=0.5.2->langchain_openai) (2023.12.25)\n",
      "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.10.0->langchain_openai) (1.2.0)\n",
      "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai) (1.0.5)\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai) (0.14.0)\n",
      "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain) (1.0.0)\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "try:\n",
    "    from google.colab import drive, userdata\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False\n",
    "\n",
    "# OpenAI Secrets\n",
    "if COLAB:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY')\n",
    "\n",
    "# Install needed libraries in CoLab\n",
    "if COLAB:\n",
    "    !pip install langchain langchain_openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pC9A-LaYhsta"
   },
   "source": [
    "# Part 1.5: Prompt Engineering\n",
    "\n",
    "Prompt engineering is a burgeoning field aimed at crafting optimal prompts to effectively utilize language models for various tasks. This discipline aids in comprehending the capabilities and limitations of large language models (LLMs) by employing different strategies to extract improved responses from foundational models, negating the need for costly and time-consuming retraining or adjustments.\n",
    "\n",
    "Distinct from fine-tuning, where a model's parameters are altered through training data to optimize performance, prompt engineering directs the pre-trained foundation model to yield more precise and relevant outputs. This result is achieved through well-crafted queries, presenting examples, breaking down questions into more straightforward steps, and employing logical reasoning.\n",
    "\n",
    "The concept of \"priming\" plays a central role in prompt engineering. In this method, we provide the model with context and a few examples of the desired outcome before receiving the actual input. This setup encourages the model to replicate the primed behavior. Users can fine-tune the model's understanding and adjust its responses to fit the conversation's context by engaging with the LLM using questions, statements, or instructions.\n",
    "\n",
    "Prompt engineering represents a significant advancement in using, developing, and comprehending language models, particularly more significant variants. It revolves around the innovative design of prompts and interactions to unlock new potentials in language technology, tackle its limitations, and glean more profound insights into its mechanisms. Through prompt engineering, we're equipped with methodologies and approaches to extend the capabilities and applications of language models.\n",
    "\n",
    "Prompt engineering emerges as the most efficient method for leveraging large language models' capabilities, refining how we engage and steer these advanced tools. This innovative field enhances the models' functionality, elevates safety standards, and deepens our comprehension of their operational framework. By integrating a variety of techniques for interaction and development, prompt engineering fosters the achievement of unprecedented milestones, such as enriching domain-specific knowledge without altering the underlying model parameters or resorting to fine-tuning. This approach emphasizes the significance of crafting superior prompts to yield higher-quality outcomes. Our discussion centers on the optimal practices for prompt engineering, mainly focusing on OpenAI models. You could apply these techniques to other models, such as those offered by Anthropic and AI21, as well as the Amazon Titan models, guiding users on maximizing their utility effectively.\n",
    "\n",
    "## Understanding Prompts\n",
    "\n",
    "We will look at four key components of most prompts. Many prompts don't require all four elements. Their form depends on the task. We'll cover more examples soon. Most prompts consist of the following parts:\n",
    "\n",
    "* **Instructions** Describes a task for the model, which can be a task description or instruction on how the model should perform.\n",
    "* **Context** External information to guide the model.\n",
    "* **Input data** The input we want a response for.\n",
    "* **Output indicator** Usually, the prompt's last part specifies that the model's output is to follow.\n",
    "\n",
    "The following is an example of a prompt requesting a product summary.\n",
    "\n",
    "```\n",
    "Summarize the following product review in a single sentence:\n",
    "Product: Smart Digital Bathroom Scale for Body Weight, Fat, BMI, Muscle Mass Composition\n",
    "Review: This scale is fantastic! The setup process was quick and simple, taking only minutes. I was able to download the app for free on my somewhat outdated iPhone 8. The feature where your weight is instantly synced with the app makes tracking effortless. The app's meal planning and calorie counting functionalities are incredibly user-friendly. I absolutely adore it! Plus, the scale has a sleek, modern appearance that is truly attractive.\n",
    "Summary:\n",
    "```\n",
    "\n",
    "This prompt might produce the following output:\n",
    "\n",
    "```\n",
    "The Smart Digital Bathroom Scale is highly praised for its quick setup, easy syncing with its app on older phones for effortless tracking, user-friendly meal planning and calorie counting features, and its sleek, modern design.\n",
    "```\n",
    "\n",
    "Figure 1.PR1 shows the components of a typical prompt.\n",
    "\n",
    "**Figure 1.PR1: Prompt Components**\n",
    "![Prompt Engineering](https://data.heatonresearch.com/images/wustl/app_genai/prompt_eng_1.jpg)\n",
    "\n",
    "## Zero-Shot Prompt Pattern\n",
    "\n",
    "Zero-shot prompting describes the technique where we present a task to an LLM without giving it further examples. We, therefore, expect it to perform the task without getting a prior look at a \"shot\" at the task. Hence, \"zero-shot\" prompting. Modern LLMs demonstrate that programmers can achieve remarkable zero-shot performance and that a positive correlation between model size and zero-shot performance.\n",
    "\n",
    "```\n",
    "Human:\n",
    "Sulfuric acid reacts with sodium chloride, and gives <chemical1>_____</chemical1> and <chemical2>_____</chemical2>:\n",
    "Assistant: the chemical1 and chemical 2 are:\n",
    "```\n",
    "\n",
    "The model might produce the following output:\n",
    "\n",
    "```\n",
    "The chemical1 is sodium sulfate (Na2SO4) and the chemical2 is hydrochloric acid (HCl).\n",
    "```\n",
    "\n",
    "\n",
    "## Few-Shot Prompt Pattern\n",
    "\n",
    "Few-shot prompting involves giving the model more information about the tasks at hand via examples. It can be used for in-context learning by providing examples of the task and the desired output. Therefore, we can condition the model on the examples to follow the task guidance more closely.\n",
    "\n",
    "You might use the following prompt to classify tweets' sentiment as positive or negative, using the Few-Shot prompt pattern.\n",
    "\n",
    "```\n",
    "Tweet: \"Received the job offer I've been waiting for. Over the moon!\" Sentiment: Positive\n",
    "Tweet: \"Passed my driving test on the first try. I'm so happy!\" Sentiment: Positive\n",
    "Tweet: \"The movie was a total waste of time. Never again.\" Sentiment: Negative\n",
    "Tweet: \"Stuck in traffic for hours. Why me?\" Sentiment: Negative\n",
    "Tweet: \"What a beautiful morning to start the day!\" Sentiment:\n",
    "```\n",
    "\n",
    "The LLM would likely continue this prompt with:\n",
    "\n",
    "```\n",
    "Positive\n",
    "```\n",
    "\n",
    "## Prompt Exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TMF-rtxgRAea",
    "outputId": "f9c5bfe1-51d7-4fd0-870b-259e9edce2eb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model response:\n",
      "The chemical1 is sodium sulfate (Na2SO4) and the chemical2 is hydrochloric acid (HCl).\n",
      "-----------\n",
      "{'token_usage': {'completion_tokens': 25, 'prompt_tokens': 65, 'total_tokens': 90}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_b28b39ffa8', 'finish_reason': 'stop', 'logprobs': None}\n"
     ]
    }
   ],
   "source": [
    "# invoke vs predict vs others .... https://python.langchain.com/docs/expression_language/interface\n",
    "\n",
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "from langchain_core.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    "    SystemMessagePromptTemplate,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "messages = [\n",
    "    SystemMessage(\n",
    "        content=\"You are a helpful assistant that.\"\n",
    "    ),\n",
    "    HumanMessage(\n",
    "        content=\\\n",
    "\"\"\"\n",
    "Human:\n",
    "Sulfuric acid reacts with sodium chloride, and gives <chemical1>_____</chemical1> and <chemical2>_____</chemical2>:\n",
    "Assistant: the chemical1 and chemical 2 are:\n",
    "\"\"\"\n",
    "    ),\n",
    "]\n",
    "\n",
    "MODEL = 'gpt-4o-mini'\n",
    "\n",
    "# Initialize the OpenAI LLM with your API key\n",
    "llm = ChatOpenAI(\n",
    "  model=MODEL,\n",
    "  #model=\"gpt-4o-mini\",\n",
    "  temperature= 0.0,\n",
    "  n= 1,\n",
    "  max_tokens= 256)\n",
    "\n",
    "print(\"Model response:\")\n",
    "output = llm.invoke(messages)\n",
    "print(output.content)\n",
    "print(\"-----------\")\n",
    "print(output.response_metadata)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.11 (genai)",
   "language": "python",
   "name": "genai"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
